Towards predicting post-editing productivity

Machine translation (MT) quality is generally measured via automatic metrics, producing scores that have no meaning for translators who are required to post-edit MT output or for project managers who have to plan and budget for transla- tion projects. This paper investigates correlations between two such automatic metrics (general text matcher and translation edit rate) and post-editing productivity. For the purposes of this paper, productivity is measured via processing speed and cognitive measures of effort using eye tracking as a tool. Processing speed, average fixation time and count are found to correlate well with the scores for groups of segments. Segments with high GTM and TER scores require substantially less time and cognitive effort than medium or low-scoring segments. Future research involving score thresholds and confidence estimation is suggested

How strongly do word reading times and lexical decision times correlate? Combining data from eye movement corpora and megastudies

We assess the amount of shared variance between three measures of visual word recognition latencies: eye movement latencies, lexical decision times and naming times. After partialling out the effects of word frequency and word length, two well-documented predictors of word recognition latencies, we see that 7-44% of the variance is uniquely shared between lexical decision times and naming times, depending on the frequency range of the words used. A similar analysis of eye movement latencies shows that the percentage of variance they uniquely share either with lexical decision times or with naming times is much lower. It is 5 – 17% for gaze durations and lexical decision times in studies with target words presented in neutral sentences, but drops to .2% for corpus studies in which eye movements to all words are analysed. Correlations between gaze durations and naming latencies are lower still. These findings suggest that processing times in isolated word processing and continuous text reading are affected by specific task demands and presentation format, and that lexical decision times and naming times are not very informative in predicting eye movement latencies in text reading once the effect of word frequency and word length are taken into account. The difference between controlled experiments and natural reading suggests that reading strategies and stimulus materials may determine the degree to which the immediacy-of-processing assumption and the eye-mind assumption apply. Fixation times are more likely to exclusively reflect the lexical processing of the currently fixated word in controlled studies with unpredictable target words rather than in natural reading of sentences or texts

The effects of changes in the order of verbal labels and numerical values on children's scores on attitude and rating scales

Research with adults has shown that variations in verbal labels and numerical scale values on rating scales can affect the responses given. However, few studies have been conducted with children. The study aimed to examine potential differences in children’s responses to Likert-type rating scales according to their anchor points and scale direction, and to see whether or not such differences were stable over time. 130 British children, aged 9 to 11, completed six sets of Likert-type rating scales, presented in four different ways varying the position of positive labels and numerical values. The results showed, both initially and 8-12 weeks later, that presenting a positive label or a high score on the left of a scale led to significantly higher mean scores than did the other variations. These findings indicate that different arrangements of rating scales can produce different results which has clear implications for the administration of scales with children

Two stages of parafoveal processing during reading: Evidence from a display change detection task

We used a display change detection paradigm (Slattery, Angele, & Rayner Human Perception and Performance, 37, 1924–1938 2011) to investigate whether display change detection uses orthographic regularity and whether detection is affected by the processing difficulty of the word preceding the boundary that triggers the display change. Subjects were significantly more sensitive to display changes when the change was from a nonwordlike preview than when the change was from a wordlike preview, but the preview benefit effect on the target word was not affected by whether the preview was wordlike or nonwordlike. Additionally, we did not find any influence of preboundary word frequency on display change detection performance. Our results suggest that display change detection and lexical processing do not use the same cognitive mechanisms. We propose that parafoveal processing takes place in two stages: an early, orthography-based, preattentional stage, and a late, attention-dependent lexical access stage

Effects of syntactic context on eye movements during reading

Previous research has demonstrated that properties of a currently fixated word and of adjacent words influence eye movement control in reading. In contrast to such local effects, little is known about the global effects on eye movement control, for example global adjustments caused by processing difficulty of previous sentences. In the present study, participants read text passages in which voice (active vs. passive) and sentence structure (embedded vs. non-embedded) were manipulated. These passages were followed by identical target sentences. The results revealed effects of previous sentence structure on gaze durations in the target sentence, implying that syntactic properties of previously read sentences may lead to a global adjustment of eye movement control

Parafoveal preview effects from word N+1 and word N+2 during reading: A critical review and Bayesian meta-analysis

The use of gaze-contingent display techniques to study reading has shown that readers attend not only to the currently fixated word, but also to the word to the right of the current fixation. However, a critical look at the literature shows that there are a number of questions that cannot be readily answered from the available literature reviews on the topic. First, there is no consensus on whether readers also attend to the second word to the right of fixation. Second, it is not clear whether parafoveal processing is more efficient in languages such as Chinese. Third, it is not well understood whether the measured effects are confounded by the properties of the parafoveal mask. The present study addressed these issues by performing a Bayesian meta-analysis of 93 experiments that used the boundary paradigm (Rayner, 1975). There were three main findings: 1) the advantage of previewing the second word to the right is modest in size and likely not centred on zero; 2) Chinese readers seem to make a more efficient use of parafoveal processing, but this is mostly evident in gaze duration; 3) there are interference effects associated with using different parafoveal masks that roughly increase when the mask is less word-like